Machine Learning Techniques to Thyroid Diagnosis and Treatment Prediction

Abstract

The thyroid is a steroid hormone that is located in the front of the neck. Its main function is to produce the thyroid gland, that is necessary for our overall health. This is a probable failure can result in thyroid hormone production that is either insufficient or excessive. As a result of one or maybe more swellings growing inside the thyroid, it might become inflamed or enlarged. Some of these nodules may harbor cancerous tumors. Sodium levothyroxine, generally defined as LT4, is a synthesized thyroid hormone used to treat thyroid problems and diseases and is one of the most commonly used medications. Predictions about therapy can help endocrinologists do their jobs better and enhance the quality of life for their patients. Numerous research has been published to date which focuses on the prognosis of thyroid illnesses based on the development of people\'s hormonal markers. This study, on the other hand, tries to forecast the LT4 therapy trend for hypothyroidism patients. A specific dataset containing clinical records on patients treated at Naples\' \"AOU Federico II\" hospital was created to achieve this purpose. Because each patient\'s whole medical history can be accessed at any moment, it was possible to forecast the duration of each patient\'s therapy based on the trend of hormonal attributes and other features analyze, in order to determine whether it should be raised or decreased. We used a variety of machine learning methods to conduct this research. In specifically, we looked at the results of ten classification techniques. The findings of the various methods are promising, specifically in the context of the Additional Classifier, which achieves an accuracy of 84 percent.

Introduction

I. INTRODUCTION

The thyroid is a tiny part of the endocrine system in the neck that produces hormones and releases them into the bloodstream. The thyroid endocrine system regulates heart rate, body temperature, and, most importantly, metabolism (the ability of the body to absorb and use food)[26]. The thyroid hormone can act abnormally (hyperthyroidism with high hormones) or abnormally (hypothyroidism with decreased hormones), resulting in serious complications [24]. In addition, the thyroid gland can become inflamed (thyroiditis) or grow as a result of one or more swellings that occur within it. Several of these nodules may harbor cancerous tumors. As a result, thyroid illness therapy is a highly important issue. Dosing levothyroxine is difficult since it depends so much on the patient's residual thyroid function, body weight, and thyroid-stimulating hormone levels .

As a result, patients' levothyroxine dosage should be adjusted throughout their lives based on sensor changes (such as weight or endocrine changes) and associated medical conditions. These necessitates an ongoing assessment of the patient's condition based on medical and laboratory findings, as well as appropriate adjustments to their levothyroxine therapy. As a result, forecasting treatment trends could be beneficial to endocrinology and can enhance the patient's quality of life. When it comes to patient tracking, endocrinologists can profit from using ML approaches. Current research has shown that categorizing and forecasting can be done successfully, and as a result, have been extensively used in the diagnosis of a variety of illnesses, including cardiovascular disease, diabetes, and Parkinson's disease [3, 4], reducing the time and costs associated with patient care.

This paper presents a method based on ML techniques that uses thyroid hormone parameters as well as other clinical data to forecast whether the patient's treatment should be increased, lowered, or kept the same. Despite previous research that tries to detect disease, the suggested technique looks at the medical symptoms of hypothyroidism patients who are taking thyroid hormone to predict therapy trends.

A selection of features identified based on endocrinologist experience comprised the proposed prediction technique.  These characteristics are utilized to train ML and a neural network classification to see how well they can predict LT4 therapy patterns in patients. The classifiers' performance is evaluated on an actual dataset created by combining data gathered from the medical system utilized at Naples' "AOU Federico II" hospital. Another contribution of this article is this dataset, which was created by preprocessing each patient's information regarding the procedure.

II. RELATED WORK

Numerous publications in the research deal with the detection of thyroid disorders utilizing hormonal tests and patient characteristics such as age and gender. ML classification and estimation techniques are used in some studies, while DNN models are used in others. The first group includes a variety of Izdihar and Bozkus [2] classifications from the UCI collection. The tree-based technique was used to diagnose thyroid illness. These created a machine learning technology, specifically for MLTDD, that can anticipate thyroid hormone abnormalities intelligently, which is a diagnostic of thyroid illnesses. According to the study, the total testing accuracy is 98.7% and the testing accuracy is 99.8%.To conduct a comparative detection of thyroid disease, the researchers in  emphasis on ML approaches such as SVM, Multiple Regression Analysis, Nave Bayes, and Decision Trees. Their findings (precision of 99.23 percent) demonstrate that decision trees always had the optimum showing and can be successfully employed as a tool for thyroid illness screening.  The authors conducted research in [16] and [5] to determine the association between TSH, T3, and T4 characteristics and hyperthyroidism or hypothyroidism utilizing different data mining techniques algorithms and to predict thyroid disease utilizing various data mining methodologies. On the data that is accessible of the UCI data archive, the authors used data mining algorithms such as KNN, Naive Bayes, SVM, and ID3.

In contrast, the authors of the study [20] employed neural systems to diagnose different forms of thyroid illness. They conducted a study on 244 patients with various diseases to determine the health of their thyroid, considering some hormonal indicators as well as the patient's age. The findings of this study reveal that the neural system provides extremely exact results, correctly categorizing thyroid diseases based on hormonal data.  The researchers are conducted a experiment with the objective of defining the two most frequent thyroid disorders, hyperthyroidism and hypothyroidism. Multivariable logistic regression techniques and neural networks were used to classify the data. The study involved 310 patients, and the models were used in this situation as well. The study involved 310 cases, and the models used demographic and hormonal data as input. In all cases, the neural network model outperformed logistic regression analysis (with an average accuracy of 91.4%). This study differs from those mentioned previously in that it is the first to aim on these therapeutic prediction of thyroid disease. As a result, the study's main goal is to use all of information data gathered on a patient across time to forecast whether LT4-based therapy should be increased or lowered. To reach this outcome, we used and assessed various ML algorithms to predict the length of treatment for people with thyroid disorders in this study. Furthermore, based on the endocrinologist's experience with the patient's treatment, this article recommends a new combination of characteristics. Finally, another edition of our research with the use of a database derived from real-world data. This dataset consists of two subs-datasets that detail the people's clinical history and current state.

III. RESEARCH METHODOLOGY

On the basis of the patient's historical and present data, the suggested approach tries to forecast the medication trend of thyroid disease. This part begins with a description of the dataset's development, followed by a description of the suggested features model, and finally a discussion of the machine learning methods employed in this research and their validation.

A. Data Collection

To carry out this research, we compiled a database of thyroid illness patients facilitated at the "AOU Federico II" Naples hospital. These data input was created by combining two data sources that contained information on 800 patients. The first data source, in particular, is the user's personal information family history, physical characteristics, and some clinical information for each patient. The logbook of the doctor's appointments, but on the other side, is the second data source. It comprises all of the information regarding the therapeutic tests and visits performed by the doctor at the time for each patient. The patient identifier is used as a key to combining the information from two sources into one huge data set. Following that, a cleaning action is carried out. All missing values and incorrect data are addressed in particular. Furthermore, while looking at each patient's clinical history, all individuals who have only had one visit are removed from the dataset because they are insufficient to analyze the disease's progression. Furthermore, we only included hyperthyroidism patients in this study since they had received the tested medication for the prediction.

Hypothyroidism is a condition that is present at birth. The three macro-pathologies detected in the dataset are Hashimoto's thyroiditis (ii), hypothyroidism (ii), and hypothyroidism (ii) (iii). The final collection contains data from 247 hypothyroid patients with an aggregate age of 46 years, most of whom had visited the hospital numerous times throughout several decades, or even multiple times in the same year in some cases. The whole data set contains 2784 instances, each of which corresponds to a unique patient and a unique session after which the relevant material and psychological parameters are collected.

B. The Proposed Mode Of Operation

The number of characteristics employed for this study is derived from a set of 135 available variables that describe the patients from various perspectives. These characteristics include the patient's biographical information, health history histories, physiological factors, hormonal and thyroid statistics, and blood test parameters. We chose 27 features from the first feature set that are associated to patient data and thyroid characteristics. An expert makes this selection based on the criteria that are commonly used to evaluate a patient's therapy. Several parameters are also excluded since they are present in only a small number of individuals and thus are missing from more than half of the dataset. Table 1 lists one attribute in the dataset for each row, having the first column containing the name, the second column including a short overview, and the third column containing the kind.

The projected features is shown at the last row: Treatment for LT4 is on the rise. This can take four various values: raised only when a prescription tends to increase, decreased when the patient requires a smaller therapy, constant when the therapy must remain constant, and others, which will be a combination of several scenarios, including when the patient requires to stop taking the medication.

C. Classifier

We employed many machine learning classifiers to predict how a patient with a thyroid condition will respond to treatment. The classifier tells the endocrinologist whether the LT4 prescription should be raised, lowered, or maintained based on the patient's history and current condition. To figure out which method best labeled every occurrence of our dataset, we compared many methods with distinct properties.

The boosting algorithms are a subset of the algorithms chosen. We employed Algorithms, Gradient Boosting, XGBC, and CatBoost in particular. These methods employ weak learners such as decision trees, are non-parametric, and do not presume or required data to follow a specific distribution. When training a grader at any layer, the Ada-boost Classifier (ABC) adds weight to every training item; each improperly rated item receives more weight, making it more likely to occur in the following grader's training subset. Because AdaBoost's performance is significantly influenced by outliers, as the algorithm seeks to modify each item appropriately, the biggest disadvantage is noisy data.

The acronym Gradient Boosting Classifier (GBC) [7] stands for Gradient Descent + Boosting. It's the same as the last one, but the difference lies in how it addresses its predecessor's flaws. Unlike AdaBoost, which changes the instance variables with each connection, this method seeks to adjust the previous predictor to the prior predictor's remaining mistakes. Gradient augmentation recasts enhancements as a quantitative optimization issue in which the goal is to use a gradient descent-like strategy to lower the nonlinear function of the model that includes poor pupils. Gradient Boosting [19] has been improved using Extreme Gradient Boosting (XGBC).

XGBoost, or eXtreme Gradient Boosting, is a supervised machine learning technique that is built on decision trees and uses a gradients augmentation framework. It is built on three important aspects that provide it optimal performance and speed: Sparse Implementation which is awareness of missing data values and handles them automatically. Parallelization of tree structures is supported by a block structure. Continuous training on new information to increase an already adapted model. As you probably have seen, Xgboost no longer moves through trees in a linear fashion; instead, predictions are necessary after each tree in order to change the gradients. Virtualization occurs at an extremely low level during the development of each tree. Every one of the tree's various branches is individually trained.

CBC (CatBoost Classifier) [11] constructs a sequence of decision trees in a sequential manner during training. When compared to earlier trees, each successive tree has a lower loss. It is designed for explanatory data, and the internal categorization of some categorical features reduces the training process significantly as compared to XGBoost, but it has more learning resources as a result of the extended training period. The decision trees [23]-based classifiers are the second category of classifiers chosen. We employed the Decision Tree, ExtraTree, and Random Forest in particular.

The categorization features are learned in the form of a tree with a Decision Tree Classifier (DCC) [23], where every other node illustrates a parameter, A leaf indicates the class's predicted number based on the ideas of the other attributes, that are represented in the network by the route from the root node to the leaf node. Decision trees are used in the Extra-Tree Classifier (EXTC) [15] and Random Forest Classifier(RFC)[9]. The input data, knot division, and speed are the three main differences between these classifiers. The random forest uses the replacement to subsample the input data, whereas the extra trees utilize the complete primary sample.

The Naive Classification Algorithm (NBC) [27] is a supervised machine learning algorithm based on Bayes' principle that predicts the likelihood of an event based on prior experience of likely situations. Its operation is based on the fact that all qualities are independent of one another, and it is composed of three steps: class probability calculation(i), conditional probability calculation (ii), and selection (iii).

The non-parametric KNN Classifier [18] makes no assumptions regarding the data distribution it examines. Its operating principle is based on the resemblance of features: the closest an example is to a piece of data, the more similar they are to the knn. Typically, the Euclidean distance is used to calculate similarity, and the procedure calls for setting a parameter k, It defines the number of observations that are closest together. Following that, the algorithm estimates the k shortest paths.

IV. PROPOSED SYSTEM

We employed a deep artificial neural network called the Multilayer Perceptron Classifier (MLPC) [6]. It is made up of several perceptrons, each with an input nodes that receive data, There are an infinite number of hidden layers connecting these two that serve as MLPs coupled to the digital engine, making judgments or projections about the input. This method was chosen in order to determine the best classifier, as well as to compare results with a previous deep learning method. The scikit-learn package, which has become one of the foremost widely used open libraries for machine learning techniques, was used to create all of the models in Python. The following are the values used to fit the classifier for each classifier:

The class value is the ratings related to classes in the form "class label: weight" in the Decision Tree Classifier(class weight='balanced', max depth=5). The equitable model utilizes the other factor to indicate the maximum capacity of the tree, in our case 5. Rather than using the possible values to instantly modify values in the input data that are inversely related to class intervals as "n samples / (n classes * np. bincount(y)), the equitable model utilizes the other factor to indicate the total depth of the tree, in our case 5. The number of neighbours to use is three in KNN(3). (class weight='balanced,' random state=1, max depth=5, n estimators=10, max features=1) Random Forest Classifier: in the same way as the decision tree, We chose a weight that is linked with balanced courses, as well as a degree of saturation of 5. Whenever the train test split method splits matrices or vectors into randomized trains and test subsets, the random phase variable ensures that the same result is obtained every time. The amount of forest trees is indicated by n estimators, and the number of characteristics to examine while looking for the optimal subdivision is indicated by max features.

In this scenario, we've decided to use the max features' qualities in each subdivision while determining an integer. GBC(n estimators=20, training rate=0.01, max features=2, max depth=2, random state=0): The amount of boost cycles to run is indicated by n estimators. A substantial percentage usually produces better outcomes since the gradients increase is strong enough to prevent over-fitting. The learning rate is a simple exponent that correlates to the pace at which the fault is rectified from one tree to the next; When looking for the best subdivision, you'll want to look at as many qualities as possible: The penetration level of the regression trees estimators is indicated by max depth. The amount of nodes of the tree is limited by the maximum depth. At each boost iteration, verifies the randomized seed supplied to each Tree estimator.

To determine the degree of precision or usefulness of any ML model, a thorough evaluation of the results achieved whenever the method is used to theoretical rate on real data is required. As a result, Cross-validation [22] is a statistical approach that is very useful for testing machine learning strategies, to generate a more trustworthy estimate of performance measures. In more detail, accuracy refers to the model's accuracy, or the percentage of the testing dataset upon which model makes a valid prediction. Accuracy is defined as follows: true positives (TP) and true negatives (TN) are examples that are correctly classified, whereas false positives (FP) and false negatives (FN) are occurrences that are misclassified:

V. RESULTS AND DISCUSSIONS

We used the results of the above-mentioned predictors, as explained in the preceding subsection, to forecast the course of a patient's treatment with levothyroxine and determine regardless of whether the prescription should be maintained, increased, or lowered. We use k-cross evaluation, as previously described with k = 10 for the evaluation of the suggested approach. The effectiveness of the classifiers is assessed in particular on all of the data inputs provided in Section 3. As a result, we present the findings of all 4 trials (one for every data input under consideration) in Tables 2, 3, 4, and 5, respectively. Each table lists the considered classifier in the first column, followed by values for accuracy, precision, recall, and F-Score.

Only two of the suggested classifiers, XGBC and CBC, could be used on the original dataset. And those are the few classifiers among those chosen that is capable of functioning with a smaller number of cases. The results are presented in Table 2. The two classes have greater precision values than all the others, with 59 cent and 71 cent, respectively, but none of the validation measures is unsatisfactory. The F-score attained, for example, is 29 cents and 34 cent, respectively. As a result, we completed the previously indicated pre-processing. Table 3 shows the interpolation and balancing results for dataset D1, with the best outcomes highlighted in bold. As you'll see, EXTC and CBC are the models with the greatest results. Their F1 score, however, is somewhere between 55 and 60%, thus they are still unsatisfactory. The findings on D2 validate the classifiers previously indicated to be best 6 after using fuzzification and equalization on the dataset. KNNC, in particular, improves across the board, achieving an F-score of 80%, while CBC follows suit, achieving a similar f-score score. Finally, including an F-score of 84 percent, EXTC receives the highest score.

The results of normalizing and balancing the dataset are shown in Table 5. The three aforementioned classifiers are also proven as the best in this scenario. The KNN algorithm achieves an F-score of 80 percent, an EXTC of 0.82 percent, and a CBC of 0.82 percent using these preprocessing procedures.

Finally, utilizing fuzzification and equilibrium as input pre-processing and Additional Trees Classification as an ML model yields the best results. With these values, we got 84 percent accuracy, 85 percent precision, 84 percent recall, and 84 percent F-Score, respectively.

VI. FUTURE WORKS

In the future, we would need to expand the quantity of data and factors evaluated in order to adequately generalize our findings. With additional data, the training approach is more likely to provide more successful classifications and a more accurate portrayal of the shown effectiveness. Finally, any tertiary thyroid concerns associated with the patient could be investigated as to whether there is an additional specialized thyroid condition that affects hypothyroidism. People who have much something than one thyroid disease at a similar period are uncommon.

Conclusion

The thyroid gland is known as our body\'s \"powerhouse,\" meaning that if something goes wrong with it, the entire body suffers. As a result, early detection of a potential malfunction is critical, Estimating the course of therapy for a hypothyroid patient, for example, can be incredibly valuable for doctors treating patients. We suggested a strategy for evaluating the medication of thyroid sickness in this study. This project \'s aim is to technique is to create an ML-based decision assistance platform for endocrinologists treating thyroid disease patients. These methods are gaining popularity in medicine, and our work can be quite beneficial, given for the excellent diagnostic prediction we have attained the particular medical setting. The proposed model, in particular, can determine the progress of a patient\'s therapy based on other data relevant to the individual being treated, making it easier for the doctor to choose the drug dosage to administer. One of the most prevalent unexpected outcomes of employing machine learning systems is that people might become unduly reliant on them, which can lead to disqualification and desensitization in the therapeutic setting. Ten alternative ML classifiers from distinct algorithm classes are used to test the proposed feature model. In fact, to test even a basic DL method, we chose models from the category of technology as a medium, modeling based on decision trees, modeling that have been heavily utilized in the research, and modeling based on a neural network. The models are put to the test on a total of 2211 instances, which correspond to a total of 247 patients. The utilization of a dataset that gathered genuine information from people undergoing medication at the Naples hospital is one of the work\'s contributions. The pre-processing of the information that would been used in the classifications is another critical stage, particularly when using the SMOTE approach, which is a minority oversampling strategy. Because the three forms are not properly represented in the original dataset, this is required. The issue is mitigated in this phase by interpolation of the minority class samples at random. The data obtained and shown in Section 4 show that the EXTC model is correct, when comparing to the other models used, performs well, with an F-score of 84 percent. The dataset\'s quality, on the alternative side, is a major flaw in this analysis, as it was constructed using actual information from hospital patients, as already mentioned.

References

[1] Spline Interpolation, pages 141–173. Springer New York, New York, NY, 2006. [2] Izdihar Al-muwaffaq and Zeki Bozkus. Mltdd : Use of machine learning techniques for diagnosis of thyroid gland disorder. 2016. [3] L. Aversano, M. L. Bernardi, M. Cimitile, and R. Pecori. Early detection of parkinson disease using deep neural networks on gait dynamics. In 2020 International Joint Conference on Neural Networks (IJCNN), pages 1–8, 2020. [4] Lerina Aversano, Mario Luca Bernardi, Marta Cimitile, and Riccardo Pecori. Fuzzy neural networks to detect parkinson disease. In 29th IEEE International Conference on Fuzzy Systems, FUZZ-IEEE 2020, Glasgow, UK, July 19-24, 2020, pages 1–8. IEEE, 2020. [5] A. Begum and A. Parkavi. Prediction of thyroid disease using data mining techniques. In 2019 5th International Conference on Advanced Computing Communication Systems (ICACCS), pages 342–345, 2019. [6] Mario Luca Bernardi, Marta Cimitile, Fabio Martinelli, and Francesco Mercaldo. Keystroke analysis for user identification using deep neural networks. In International Joint Conference on Neural Networks, IJCNN 2019 Budapest, Hungary, July 14-19, 2019, pages 1–8. IEEE, 2019. [7] G. Biau, B. Cadre, and L. Rouvie`re. Accelerated gradient boosting. Machine Learning, 108(6):971–992, 2019. [8] Shiva Borzouei, Hossein Mahjub, NegarAsaad Sajadi, and Maryam Farhadian. Diagnosing thyroid disorders: Comparison of logistic regression and neural network models. Journal of Family Medicine and Primary Care, 9:1470, 06 2020. [9] Leo Breiman. Random forests. Machine Learning, 45(1):5–32, 2001. [10] Nitesh V. Chawla, Kevin W. Bowyer, Lawrence O. Hall, and W. Philip Kegelmeyer. Smote: Synthetic minority over-sampling technique. J. Artif. Int. Res., 16(1):321357, June 2002.

Copyright

Copyright © 2022 Chandrika B N, Sindhu D, Dr. Ravikumar G K. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.